Semi-Supervised Learning with Sparse Autoencoders in Automatic Speech Recognition

This work is aimed at exploring semi-supervised learning techniques to improve the performance of Automatic Speech Recognition systems. Semi-supervised learning takes advantage of unlabeled data in order to improve the quality of the representations extracted from the data. The proposed model is a neural network where the weighs are updated by minimizing the weighted sum of a supervised and an unsupervised cost function, simultaneously. Those costs are evaluated on the labelled and unlabeled portions of the data set, respectively. The combined cost is optimized through mini-batch stochastic gradient descent via standard backpropagation. The model was tested on a phone classification task on the TIMIT American English data set and on a written digit classification task on the MNIST data set. Our results show that the model outperforms a network trained with standard backpropagation on the labelled material alone. The results are also in line with state-of-the-art graph-based semi-supervised training methods.